SherbyNet - /page/wikibook/entry.mu

Micron Document

<!DOCTYPE html>

<title>Native Command Queuing</title>

</head>

Native Command Queuing

</h1>

</header>

In <a href="Computing" title="Computing">computing</a>, Native Command Queuing (NCQ) is an extension of the <a href="Serial_ATA" class="mw-redirect" title="Serial ATA">Serial ATA</a> protocol allowing <a href="Hard_disk_drive" title="Hard disk drive">hard disk drives</a> to internally optimize the order in which received read and write commands are executed. This can reduce the amount of unnecessary drive head movement, resulting in increased performance (and slightly decreased wear of the drive) for workloads where multiple simultaneous read/write requests are outstanding, most often occurring in <a href="Server_(computing)" title="Server (computing)">server</a>-type applications.

<div class="mw-heading mw-heading2"><h2 id="History">History</h2></div>

Native Command Queuing was preceded by <a href="Parallel_ATA" title="Parallel ATA">Parallel ATA</a>'s version of <a href="Tagged_Command_Queuing" title="Tagged Command Queuing">Tagged Command Queuing</a> (TCQ). ATA's attempt at integrating TCQ was constrained by the requirement that ATA host bus adapters use <a href="Industry_Standard_Architecture" title="Industry Standard Architecture">ISA</a> bus device protocols to interact with the <a href="Operating_system" title="Operating system">operating system</a>. The resulting high CPU overhead and negligible performance gain contributed to a lack of market acceptance for ATA TCQ.

NCQ differs from TCQ in that, with NCQ, each command is of equal importance, but NCQ's host bus adapter also programs its own <a href="Direct_memory_access#Bus_mastering" title="Direct memory access">first party DMA</a> engine with CPU-given <a href="Direct_memory_access" title="Direct memory access">DMA</a> parameters during its command sequence whereas TCQ interrupts the CPU during command queries and requires it to modulate the ATA host bus adapter's <a href="Direct_memory_access#Third-party" title="Direct memory access">third party DMA</a> engine. NCQ's implementation is preferable because the drive has more accurate knowledge of its performance characteristics and is able to account for its rotational position. Both NCQ and TCQ have a maximum queue length of 32 outstanding commands.<a href="#cite_note-sata-1">[1]</a><a href="#cite_note-pata-7Vol1-2">[2]</a> Because the ATA TCQ is rarely used, <a href="Parallel_ATA" title="Parallel ATA">Parallel ATA</a> (and the IDE mode of some chipsets) usually only support one outstanding command per port.

For NCQ to be enabled, it must be supported and enabled in the SATA host bus adapter and in the hard drive itself. The appropriate driver must be loaded into the operating system to enable NCQ on the host bus adapter.<a href="#cite_note-3">[3]</a>

Many newer chipsets support the <a href="Advanced_Host_Controller_Interface" title="Advanced Host Controller Interface">Advanced Host Controller Interface</a> (AHCI), which allows operating systems to universally control them and enable NCQ. <a href="DragonFly_BSD" title="DragonFly BSD">DragonFly BSD</a> has supported AHCI with NCQ since 2.3 in 2009.<a href="#cite_note-4">[4]</a><a href="#cite_note-5">[5]</a> <a href="Linux_kernel" title="Linux kernel">Linux kernels</a> support AHCI natively since version 2.6.19, and <a href="FreeBSD" title="FreeBSD">FreeBSD</a> fully supports AHCI since version 8.0. <a href="Windows_Vista" title="Windows Vista">Windows Vista</a> and <a href="Windows_7" title="Windows 7">Windows 7</a> also natively support AHCI, but their AHCI support (via the msahci service) must be manually enabled via registry editing if controller support was not present during their initial install. <a href="Windows_7" title="Windows 7">Windows 7</a>'s AHCI enables not only NCQ but also <a href="Trim_(computing)" title="Trim (computing)">TRIM</a> support on <a href="SSD" class="mw-redirect" title="SSD">SSD</a> drives (with their supporting <a href="Firmware" title="Firmware">firmware</a>). Older operating systems such as <a href="Windows_XP" title="Windows XP">Windows XP</a> require the installation of a vendor-specific driver (similar to installing a <a href="RAID" title="RAID">RAID</a> or <a href="SCSI" title="SCSI">SCSI</a> controller) even if AHCI is present on the host bus adapter, which makes initial setup more tedious and conversions of existing installations relatively difficult as most controllers cannot operate their ports in mixed AHCI–SATA/IDE/legacy mode.

<div class="mw-heading mw-heading2"><h2 id="Hard_disk_drives">Hard disk drives</h2></div>

<div class="mw-heading mw-heading3"><h3 id="Performance">Performance</h3></div>

A 2004 test with the first-generation NCQ drive (Seagate 7200.7 NCQ) found that while NCQ increased IOMeter performance, desktop application performance decreased.<a href="#cite_note-6">[6]</a> One review in 2010 found improvements on the order of 9% (on average) with NCQ enabled in a series of Windows multitasking tests.<a href="#cite_note-7">[7]</a>

NCQ can negatively interfere with the operating system's <a href="I/O_scheduler" class="mw-redirect" title="I/O scheduler">I/O scheduler</a>, decreasing performance;<a href="#cite_note-young-8">[8]</a> this has been observed in practice on <a href="Linux" title="Linux">Linux</a> with <a href="RAID-5" class="mw-redirect" title="RAID-5">RAID-5</a>.<a href="#cite_note-9">[9]</a> There is no mechanism in NCQ for the host to specify any sort of deadlines for an I/O, like how many times a request can be ignored in favor of others. In theory, a queued request can be <a href="Resource_starvation" class="mw-redirect" title="Resource starvation">delayed by the drive an arbitrary amount of time</a> while it is serving other (possibly new) requests under I/O pressure.<a href="#cite_note-young-8">[8]</a> Since the algorithms used inside drives' firmware for NCQ dispatch ordering are generally not publicly known, this introduces another level of uncertainty for hardware/firmware performance. Tests at Google around 2008 have shown that NCQ can delay an I/O for up to 1–2 seconds. A proposed workaround is for the operating system to artificially starve the NCQ queue sooner in order to satisfy low-latency applications in a timely manner.<a href="#cite_note-10">[10]</a>

On some drives' firmware, such as the <a href="Western_Digital_Raptor" title="Western Digital Raptor">WD Raptor</a> circa 2007, <a href="Disk_buffer" title="Disk buffer">read-ahead</a> is disabled when NCQ is enabled, resulting in slower sequential performance.<a href="#cite_note-11">[11]</a>

SATA <a href="Solid-state_drive" title="Solid-state drive">solid-state drives</a> profit significantly from being able to queue multiple commands for parallel workloads. For PCIe-based <a href="NVMe" class="mw-redirect" title="NVMe">NVMe</a> SSDs, the queue depth was even increased to support a maximum of 65,535 queues with up to 65,535 commands each.

<div class="mw-heading mw-heading3"><h3 id="Safety_(FUA)">Safety (FUA)</h3></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hatnote{font-style:italic}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output .hatnote i{font-style:normal}.mw-parser-output .hatnote+link+.hatnote{margin-top:-0.5em}@media print{body.ns-0 .mw-parser-output .hatnote{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div role="note" class="hatnote navigation-not-searchable">See also: <a href="Disk_buffer#Force_Unit_Access_(FUA)" title="Disk buffer">Disk buffer § Force Unit Access (FUA)</a></div>

One lesser-known feature of NCQ is that, unlike its ATA TCQ predecessor, it allows the host to specify whether it wants to be notified when the data reaches the disk's platters, or when it reaches the disk's buffer (on-board cache). Assuming a correct hardware implementation, this feature allows data consistency to be guaranteed when the disk's on-board cache is used in conjunction with system calls like <a href="Fsync" class="mw-redirect" title="Fsync">fsync</a>.<a href="#cite_note-12">[12]</a> The associated write flag, which is also borrowed from SCSI, is called <a href="Force_Unit_Access" class="mw-redirect" title="Force Unit Access">Force Unit Access</a> (FUA).<a href="#cite_note-Smith2010-13">[13]</a><a href="#cite_note-14">[14]</a><a href="#cite_note-15">[15]</a>

<div class="mw-heading mw-heading2"><h2 id="Solid-state_drives">Solid-state drives</h2></div>

NCQ is also used in newer <a href="Solid-state_drive" title="Solid-state drive">solid-state drives</a> where the drive encounters latency on the host, rather than the other way around. For example, <a href="Intel" title="Intel">Intel</a>'s X25-E Extreme solid-state drive uses NCQ to ensure that the drive has commands to process while the host system is busy processing CPU tasks.<a href="#cite_note-TechRep_SSD_NCQ-16">[16]</a>

NCQ also enables the SSD controller to complete commands concurrently (or partly concurrently, for example using pipelines) where the internal organisation of the device enables such processing.

The <a href="NVM_Express" title="NVM Express">NVM Express</a> (NVMe) standard also supports command queuing, in a form optimized for SSDs.<a href="#cite_note-ahci-nvme-17">[17]</a> NVMe allows multiple queues for a single controller and device, allowing at the same time much higher depths for each queue, which more closely matches how the underlying SSD hardware works.<a href="#cite_note-18">[18]</a>

<ul><li><a href="I/O_scheduling" title="I/O scheduling">I/O scheduling</a></li>

<li><a href="Write_barrier" title="Write barrier">Write barrier</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist reflist-columns references-column-width" style="column-width: 30em;">

<li id="cite_note-sata-1"><a href="#cite_ref-sata_1-0">^</a> <a rel="nofollow" class="external text" href="https://web.archive.org/web/20101218131508/http://seagate.com/content/pdf/whitepaper/D2c_tech_paper_intc-stx_sata_ncq.pdf">PDF white paper on NCQ from Intel and Seagate</a>

</li>

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://web.archive.org/web/20120206032939/http://www.t13.org/Documents/UploadedDocuments/docs2004/d1532v1r4b-ATA-ATAPI-7.pdf">"Volume 1 of the final draft of the ATA-7 standard"</a> (PDF). Archived from <a rel="nofollow" class="external text" href="http://www.t13.org/Documents/UploadedDocuments/docs2004/d1532v1r4b-ATA-ATAPI-7.pdf">the original</a> (PDF) on 2012-02-06. Retrieved 2007-01-02.</cite>

</li>

<li id="cite_note-3"><a href="#cite_ref-3">^</a> <a rel="nofollow" class="external text" href="http://download.intel.com/support/chipsets/imsm/sb/sata2_ncq_overview.pdf">"SATA II Native Command Queuing Overview", Intel Whitepaper, April 2003. </a>

</li>

<li id="cite_note-4"><a href="#cite_ref-4">^</a> <cite id="CITEREFMatthew_Dillon2009" class="citation web cs1"><a href="Matthew_Dillon" title="Matthew Dillon">Matthew Dillon</a> (2009-06-04). <a rel="nofollow" class="external text" href="http://www.dragonflybsd.org/mailarchive/kernel/2009-06/msg00004.html">""Re: DragonFly-2.3.1.165.g25822 master sys/dev/disk/ahci Makefile TODO ahci.c ahci.h ahci_attach.c ahci_cam.c ahci_dragonfly.c ahci_dragonfly.h atascsi.h""</a>.</cite>

</li>

<li id="cite_note-5"><a href="#cite_ref-5">^</a> <cite id="CITEREFMatthew_Dillon2009" class="citation web cs1"><a href="Matthew_Dillon" title="Matthew Dillon">Matthew Dillon</a> (2009). <a rel="nofollow" class="external text" href="http://bxr.su/d/share/man/man4/ahci.4">"ahci(4) — Advanced Host Controller Interface for Serial ATA"</a>. BSD Cross Reference. <a href="DragonFly_BSD" title="DragonFly BSD">DragonFly BSD</a>.</cite>

<ul><li><cite class="citation book cs1">"ahci - Advanced Host Controller Interface for Serial ATA". <a rel="nofollow" class="external text" href="http://mdoc.su/d/ahci.4">DragonFly On-Line Manual Pages</a>.</cite></li></ul>

</li>

<li id="cite_note-6"><a href="#cite_ref-6">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="http://techreport.com/review/7750/seagate-barracuda-7200-7-ncq-hard-drive/13">"Seagate's Barracuda 7200.7 NCQ hard drive - The Tech Report - Page 13"</a>. The Tech Report. 17 December 2004. Retrieved 2014-01-11.</cite>

</li>

<li id="cite_note-7"><a href="#cite_ref-7">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="http://techreport.com/review/8624/multitasking-with-native-command-queuing/5">"Multitasking with Native Command Queuing - The Tech Report - Page 5"</a>. The Tech Report. 3 August 2005. Retrieved 2014-01-11.</cite>

</li>

<li id="cite_note-young-8">^ <a href="#cite_ref-young_8-0">a</a> <a href="#cite_ref-young_8-1">b</a> <cite id="CITEREFYuShinEomYeom2010" class="citation journal cs1">Yu, Y. J.; Shin, D. I.; Eom, H.; Yeom, H. Y. (2010). "NCQ vs. I/O scheduler". ACM Transactions on Storage. 6: 1–37. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F1714454.1714456">10.1145/1714454.1714456</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:14414608">14414608</a>.</cite> <a rel="nofollow" class="external autonumber" href="http://www.cs.albany.edu/~sdc/CSI500/Fal10/DiskArmSchedulingPapers/a2-yu.pdf">[1]</a> <a rel="nofollow" class="external text" href="https://web.archive.org/web/20140111152130/http://www.cs.albany.edu/~sdc/CSI500/Fal10/DiskArmSchedulingPapers/a2-yu.pdf">Archived</a> 2014-01-11 at the <a href="Wayback_Machine" title="Wayback Machine">Wayback Machine</a>

</li>

<li id="cite_note-9"><a href="#cite_ref-9">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="http://serverfault.com/questions/305890/poor-linux-software-raid-5-performance-with-ncq">"hard drive - Poor Linux software RAID 5 performance with NCQ"</a>. Server Fault. Retrieved 2014-01-11.</cite>

</li>

<li id="cite_note-10"><a href="#cite_ref-10">^</a> Gwendal Grignou, NCQ Emulation, FLS'08 <a rel="nofollow" class="external text" href="https://www.usenix.org/legacy/publications/login/2008-06/openpdfs/lsf08reports.pdf">talk summary (p. 109)</a> <a rel="nofollow" class="external text" href="https://www.usenix.org/legacy/event/lsf08/tech/IO_grignou.pdf">slides</a>

</li>

<li id="cite_note-11"><a href="#cite_ref-11">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://lkml.org/lkml/2007/4/3/159">"Mark Lord: Re: Lower HD transfer rate with NCQ enabled?"</a>. LKML. 2007-04-03. Retrieved 2014-01-11.</cite>

</li>

<li id="cite_note-12"><a href="#cite_ref-12">^</a> <cite id="CITEREFMarshall_Kirk_McKusick" class="citation web cs1"><a href="Marshall_Kirk_McKusick" title="Marshall Kirk McKusick">Marshall Kirk McKusick</a>. <a rel="nofollow" class="external text" href="http://queue.acm.org/detail.cfm?id=2367378">"Disks from the Perspective of a File System - ACM Queue"</a>. Queue.acm.org. Retrieved 2014-01-11.</cite>

</li>

<li id="cite_note-Smith2010-13"><a href="#cite_ref-Smith2010_13-0">^</a> <cite id="CITEREFGregory_Smith2010" class="citation book cs1">Gregory Smith (2010). <a rel="nofollow" class="external text" href="https://archive.org/details/postgresqlhighpe00smit">PostgreSQL 9.0: High Performance</a>. Packt Publishing Ltd. p. <a rel="nofollow" class="external text" href="https://archive.org/details/postgresqlhighpe00smit/page/n98">78</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-84951-031-8</bdi>.</cite>

</li>

<li id="cite_note-14"><a href="#cite_ref-14">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://web.archive.org/web/20040517234919/http://www.seagate.com:80/docs/pdf/whitepaper/D2c_tech_paper_intc-stx_sata_ncq.pdf">"Native Command Queuing"</a> (PDF). Archived from <a rel="nofollow" class="external text" href="http://www.seagate.com/docs/pdf/whitepaper/D2c_tech_paper_intc-stx_sata_ncq.pdf">the original</a> (PDF) on 2004-05-17.</cite>

</li>

<li id="cite_note-15"><a href="#cite_ref-15">^</a> <cite id="CITEREFJonathan_Corbet2010" class="citation web cs1">Jonathan Corbet (2010-08-18). <a rel="nofollow" class="external text" href="https://lwn.net/Articles/400541/">"The end of block barriers"</a>. <a href="LWN.net" title="LWN.net">LWN.net</a>. Retrieved 2015-06-27.</cite>

</li>

<li id="cite_note-TechRep_SSD_NCQ-16"><a href="#cite_ref-TechRep_SSD_NCQ_16-0">^</a> <cite id="CITEREFGasior2008" class="citation web cs1">Gasior, Geoff (November 23, 2008). <a rel="nofollow" class="external text" href="http://techreport.com/articles.x/15931/1">"Intel's X25-E Extreme solid-state drive - Now with single-level cell flash memory"</a>. <a href="Tech_Report" class="mw-redirect" title="Tech Report">Tech Report</a>.</cite>

</li>

<li id="cite_note-ahci-nvme-17"><a href="#cite_ref-ahci-nvme_17-0">^</a> <cite id="CITEREFDave_Landsman2013" class="citation web cs1">Dave Landsman (2013-08-09). <a rel="nofollow" class="external text" href="https://www.sata-io.org/sites/default/files/documents/NVMe%20and%20AHCI%20as%20SATA%20Express%20Interface%20Options%20-%20Whitepaper_.pdf">"AHCI and NVMe as Interfaces for SATA Express Devices – Overview"</a> (PDF). <a href="SATA-IO" class="mw-redirect" title="SATA-IO">SATA-IO</a>. Retrieved 2013-10-02.</cite>

</li>

<li id="cite_note-18"><a href="#cite_ref-18">^</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="http://www.nvmexpress.org/about/nvm-express-overview/">"NVM Express Overview"</a>. nvmexpress.org. Retrieved 2014-11-26.</cite>

</li>

</ol></div>

<div class="mw-heading mw-heading2"><h2 id="External_links">External links</h2></div>

<ul><li><a rel="nofollow" class="external text" href="https://web.archive.org/web/20110713232900/http://www.lostcircuits.com/mambo//index.php?option=com_content&task=view&id=50&Itemid=46">Serial ATA and the 7 Deadly Sins of Parallel ATA</a> at Lost Circuits (for TCQ)</li>

<li><a rel="nofollow" class="external text" href="http://www.tomshardware.com/reviews/command-queuing-turbo-charge-sata,922.html">Hype vs. Reality</a> analysis at <a href="Tom's_Hardware" title="Tom's Hardware">Tom's Hardware</a></li></ul></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-06-24" href="https://en.wikipedia.org/wiki/?title=Native_Command_Queuing&oldid=1297147970">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>